MOTION PICTURE AND VIDEOTAPE EDITING AMD SOUND CONTROL SYSTEM 

Abstract of the Disclosure 

Means Including both method and apparatus for editing 
motion pictures and videotapes to a partially completed or flnlshec 
state, and synchronizing and purifying their sound traoks. Footage 
to be edited Is kept In a storage means, and brought by a transport 
from said storage means to a presentation means which transmits 
the Image to a viewing means? footage address notation means, by 
which the viewer may make note of points within the footage of 
interest to him? address storage means, wherein are stored footage 
addresses? address association means, by which footage addresses 
stored In said address storage means may be associated with other 
data? text Input means, by which text may be emplaced in the systen 
and linked to footage addresses by said address association means? 
graphical Input means, by whloh graphical symbolism may be emplaoea 
in the system and linked to footage addresses by said address 
association means? selection Input means, by which said text and 
graphics linked to footage addresses may be selected? footage 
rearrangement means, by which sequences seleoted by the user may 
be rearranged for viewing? sonic analysis means, analyzing the 
soundtraok of the footage Into sonic components? sonlo re-tlmlng 
means, re-tlmlng said sonic components In accordance with some 
timing signal? sonic reeonstltutlve means, reperforming the sound 
according to the new timing? sonic component visual display means, 
displaying said sonic components and other features of the sound¬ 
traok, and sonic component display modification means, permitting 
graphical modification of said sonic components disopvered and 
stored by said sonlo analysis and storage means, and other 
graphical modifications to the soundtraok. 





Background of the Invention 

This Invention relates to the art of motlon-ploture and 
videotape production, and more particularly to an Improved method 
and apparatus for editing the created footage into Intermediate 
collections of shots or finished films or videotape shows. 

It la to he understood that the terms "motion picture," 
"film" or "videotape" as used In this application are intended to 
refer not only to common motion pictures oomposed of Image frames 
on sprocketed film, or to the oommon magnetlc-oxlde videotape 
recordings, hut to any form of recording information to he present-| 
ed to a final viewer sequentially as Images, whether stored photo¬ 
graphically, electronically, or hy other means. 

It Is to he understood that the term "footage" as used 
in this application means a quantity of such film or videotape, 
or the contents of such a quantity of film or videotape. 

It is to he understood that the term "shot" as used In 
this application means a oonseoutive segment of film or videotape. 

In the production of motion-picture and videotape photo¬ 
plays and shows, persons producing the work cannot oonflne their 
production activity to exact pre-planned scenes and shots, hut 
create incidentally unwanted footage and shots of uncertain final 
use. The task of editing oonsists of discarding definitely unwanted 
footage and effecting trial combinations from among the shots con¬ 
sidered usable, viewing these combinations and selecting from 
among them, and aggregating such combinations until a production 
is completed. 

Although It is possible to envision roughly the nature oi| 
a planned sequence of known shots, unexpected flaws and virtues 
are frequently discovered upon viewing suoh sequences. Thus It Is 
desirable for the editor to he enabled to plan quickly a new 
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sequence of footage, and for him to view such sequence as promptly 
and effortlessly as possible. 

In the past, editing has been effected through the use 
of hand-operated transport means (rewinds) and viewing machines 
(viewers), together with the physical operation by the editor of 
outtlng and spllolng devices to join physloally strips of film into 
a sequence. If he desires to see the shots in another combination, 
the editor must then either re-separate the entire sequence and 
reassemble in that new combination, or make individual changes in 
the existing combination until it satisfies him. 

In these activities the editor must devote much attention 
to the storage and location of the footage which Interests him. 
Usually he must keep written numerical records of the locations of 
the shots which he thinks he may use, with respeot both to what 
reel a shot is on and at what position on the reel (generally in 
terms of feet) it is to be found. The management of these physical 
shot locations (or, as we shall call them, "addresses") occupies 
much of his attention. 

A related problem is that of recording and synchronizing 
sound tracks to motion pictures and prearrangement of synchroniza¬ 
tion for recordings to be thus synchronized. Good quality speech 
recording synchronized with actors' lip movement is ordinarily best 
attainable with expensive recording equipment electrically linked 
to the motion picture oamera or speed controlled by precision means 
under recording circumstances requiring hushed surroundings. 
Recordings made with neither electrical connection to the oamera 
nor precision speed oontrol tend not to be correctly synchronized. 
Recordings made under poor recording circumstances tend to have 
undesirable background noises. 

An alternative, called "wild" recording, is the creation 
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of a sound recording by actors watching the film, who attempt by 
mimicry to synchronize their speech with the deploted lip movement. 
The resulting recordings usually synchronize poorly to the film, 
due to the difficulty of acting with realism while attempting to 
match the pre-existent timing. 

Summary of the Invention 

It is with the above problems and considerations in mind 
that the present improved editing system has been evolved, per¬ 
mitting the editing of films and videotapes without extraneous 
physical operations or attention by the user to numerical informa¬ 
tion, permitting the concentration of the editor on aspects of the 
task relevant to his decision processes, in particular the oontents 
of the shots and transitions among them. 

It is, accordingly, among the primary objects of this 
invention to provide a method of editing in which the editor is 
freed from mechanical operations and concerns of storage and looa- 

I 

tlon, and permitted to deal with the symbolic manipulation of con¬ 
tents whioh are of editorial concern. 

Another object of the invention is to provide a machine 
or system for editing which requires no concern with numerical 
addresses and permits operation by the editor through the symbolio 
manipulation of contents. 

Another object of the invention is to perfect the syn¬ 
chronization of wild sound and thus render unnecessary simultaneous 
recording. 

Another object of the invention is to improve the quality 
of sound recordings made with inferior equipment or in noisy sur¬ 
roundings in order to simplify the creation of an acceptable 
soundtrack. 

Another objeot of the invention is to provide a simpli¬ 
fied faoility for audio mixing, l.e., combining several soundtracks 
while ohanglng volume levels into a single soundtrack with all 
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sounds In appropriate proportion. 

These and other objeots of the Invention, whioh will be¬ 
come hereafter more apparent, are aohieved by storing the footage 
on videotape or other storage means, permitting an editor to con¬ 
trol its movement through a presentation means while he views it 
on viewing means, suoh as a T7 screen, and permitting him to 

Wilkin m »hA. k'— (■» 

annotate these points or stretches of footage between them. In 
aocordanoe with the invention, the points in the footage are 
identified by some signalling means such as a pushbutton, and the 
addresses of the locations of these points on the videotape are 
then stored in a location storage means, such as a computer disk 
memory. The user may then input to the oomputer strings of des¬ 
criptive or mnemonlo text, whioh are coupled to the footage add¬ 
resses through an address association means, such as a linked list 
on computer disk storage. Through a text presentation means, such 
as a oomputer display coupled to the television screen, the text 
descriptions or mnemonics are presented again on a viewing means, 
and the user chooses the desired sequence through a selection 
input means, such as a light pen. The user may likewise use sym¬ 
bols or diagrams for the designation of shots and sequences he 
desires. In‘accordance with the Invention, the shots thus desig¬ 
nated, or copies of the shots, are automatically rearranged for 
viewing by a footage rearrangement means, such as four videotape 
recorders controlled by a general-purpose digital oomputer with a 
sorting program. The user then views the completed series of shot£ 
and may then make any desired change in the sequence with said 
selection input means, or change the text descriptions as desired. 

Points which have been designated in the footage may be 
used as synchronization points for the sound traok, which has been 
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broken Into Its constituent sonic components by a sonic analyzer. 

These constituents may also be selectively removed or 
strengthened as guided by user manipulation of the visual display 
to remove noise and improve clarity and effeot. 

A feature of the Invention resides In the fact that the 
user Is unaware of the place of storage of a given shot. 

Another feature of the Invention resides In the fact 
that the user does not need to use any numbers or coded abbrevia¬ 
tions, and may use only full English text as descriptions to con¬ 
trol the sequencing of desired shots. 

Another feature of the invention resides in the fact that 
the user may restore to Its original length a shot which he 
previously has shortened. 

Another feature of the invention resides In the faot that 
the user may effect fine trim between shots, and audio mixing, 
by manipulation of diagrams. 

Another feature of the invention resides In the faot that 
extraneous details of exercising this craft, such as physical 
manipulation of rewinds and splicers, and the numerical annotation 
Of film addresses, may be eliminated from the mental field of the 
working editor. 

Another feature of the Invention resides in the ability 
of the editor to obtain synchronization between photographed lip- 
movement and uncontrolled recordings. 

Another feature of the Invention resides In the ability 
of the editor to remove background noises which are visually 
discernible In a visual display of sonlo components of reoorded 
sound, and Improve sound character and clarity by the same meohanis 


m 
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Brief Description of the Drawing 

j 

The speolflo details of apparatus suitable for practicing 
the method of this invention, and their mode of functioning, will 
he described In full, clear, concise, and exact terms in conjunc¬ 
tion with the accompanying drawing, wherein: 

FIG. 1 Is a schematic perspective view of an editor's 
oonsole, showing viewing screen and oontrols. 

FIG. 2 is a top plan view of a function button panel, 
showing the different labelled function buttons. 

FIG. 3 Is a schematic drawing of the major parts and 
signal paths In the complete editing system. 

FIG* 4 Is a schematic view of the screen area showing a 
simulation of a list of shot descriptions as seen by the editor on 
a television soreen. 

FIG. 5 is a schematic view of the soreen area showing a 
simulation of a list of shot descriptions and an assembly list onto 
whloh one shot description has been transferred. 

FIG. 6 Is a schematic view of the soreen area showing a 
simulation of a combined display of script and shot diagrams as 
seen by the editor on a television screen. 

FIG. 7 Is a schematic view of the soreen area showing the 
simulation of a trim control diagram as seen by an editor on a 
television screen. 

FIG. 8 is a simulation of the sonic constituent display, 
showing the display of an array of sonlo components as seen by the 
editor on a television screen. 

FIG. 9 Is a simulation of the sync map,- showing the now 
line, volume contour and sync points. 

FIG. 10 is a simulation of the audio mix map as seen by 
an editor on a television screen. 




- 7 - 






Description of the Preferred Embodiment 

Referring now more particularly to the drawings, like 
numerals In the various figures will he employed to designate like 
parts. 

As seen In PIS. 1, the user's apparatus for practicing 
the instant Invention Is shown In connection with a conventional 
television screen 1. Facing the user In front of the television 
screen are an array of function buttons 2, a oulng pushbutton 3, 
forward-and-back control wheel containing binary shaft enooder 4, 
a typewriter keyboard 5» and a light pen 6. 

As seen In FIG. 2, the user selects functions from among 
various pushbuttons, 7 through 15« Their respective uses will be 
described In conjunction with speolfio operations. 

As seen In FIG. 3, the user's oontrols 18 are interfaced 
by Input lines 19 to a general-purpose digital computer 20, such ac 
a Digital Equipment Corporation PDP-8 or IBM System/360 mod 92, 
having a magnetlo disk memory 21. The user's controls are inter¬ 
faced along input lines 19 as Input devices In the standard manner] 
as will be appreciated by those skilled in the art, In such a way 
that the user's actions upon them will be transmitted to the com¬ 
puter in the form of binary information In respective oodes for 
those units. Videotape recorders or VTRs 22, 23, 24, 25, 26 are 
likewise Interfaced with the computer by output lines 27, 28, 29, 
30, 31 In such manner that signals from the computer may cause 
them to start playing, stop playing, start recording, and stop 
recording. The computer Is also linked to switching and attenuator 
matrix 32 by control line 33, In such manner that signals from 
the computer may command Input to any VTR through Input lines 34, 

Odtptfl 

35» 36, 37 » 38 and^the output lines 39» 40, 4l, 42 , 43 of any VTR 
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to 130 connected to the input of any VTR or to the user television 
screen 44. The oomputer 20 also has input lines 45 , 46 , 47 , 48 , 4; 
through which it reoeives information from VTRs 22-26 telling the 
position of their tapes. The computer is also interfaced by means 
of analog-to-dlgltal converters (not shown) to attenuator controls 
(not shown) whloh permit the oomputer to direct volume and bright¬ 
ness settings of the VTR outputs to any destination. Audio output 
from VTRs 22-26 may also be directed to user loudspeaker 50» 

The computer is also Interfaced to an Instrumentation 
tape recorder and audio reoorder 51* 52 by means of output lines 
53 > 54 and input lines 55* 56 whloh permit the computer to control 
the units and ascertain the position of the‘£tapes* Signals repre¬ 
senting the sonic outputs of these units travel on output lines 
57 » 58 to switch matrix 32 where they may be combined as output to 
user loudspeaker 50 or routed to input lines 34-38 of VTRs 22-26 
or their own input lines 59> 60. Audio leaving switch matrix 32 
may also be directed by the computer 20 on line 6l to sonic ana.lyz< 
62, where it is converted to its constituent frequencies whose 
measurements pass in digital form on line 63 to computer 20, which 
stores them on disk 21 or displays their representation, with or 
without modifications on soreen 44 by means of computer-to-vldeo 
display 64, or sends them to sonic restructuring unit 65 from 
which they may be heard as reconstituted sound from user loudspeak* 
66. The reconstituted sound may also go via line 67 to switch 
matrix 32 and thence to Inputs of VTRs 22-26. 

To provide for the automatic finding and control of sig¬ 
nificant portions Of stored motion pictures and videotapes, the 
addresses of points of Interest in videotapes mounted on VTRs 22-2< 
are transmitted from VTRs 22-26 through address lines 45-49 to 
computer 20. Such addresses are when required stored in disk 
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memory 21 in a format illustrated below. This is the point looatio 
list. The format Indicates the method of listing in digital stor¬ 
age the location addresses of event-points seleoted within a series 
of shots. 


n 


VIDEOTAPE POINT ADDRESS LIST 


point's position in list 


videotape address 


10 


(n+1 

(n+2 




videotape address of point n 
videotape address of point n+1 
videotape address of point n+2 


15 


e 


e 


e 


e 


20 


25 


A point in the videotape to be designated in the computer program 
as point n has its videotape address stored in location n of said 
list, its successor the address of point n+1 in the film is stored 
in location m-1 . etc. 

To provide for the labelling of portions of film by 
written information, means is required for storing and indexing 
said written information on computer disk 21. This means is illus¬ 
trated below, in a format which represents the form of indexing 
textual information in digital storage. 


TEXT-STRING LIST 


point's position 
in list 


start 


ADDRESS OF TEXT STRING 


end 


30 


35 




(n) 

(n+l) 




address of start 
of text string n 
address of start 
of text string 
n-KL 


address of end 
of text string 
address of end 
of text string 
n+-l 
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The aotual textual Information is stored elsewhere in an undivided 
text area on the disk* Strings of text charaoters are denoted by 
a starting point representing the starting addresses of said text 
strings on the disk and endpoints representing the end addresses 
of said text strings on the disk. The nth location in said list 
stores the starting points and endpoints of the string to he known 
by the computer as string number n, the (ntl) th location in said 
list stores the starting points and endpoints of the string to be 
known by the oomputer as string number n+1 . etc. 

BO provide a means for denoting a shot or portion of 
videotape and linking It to a textual name or description, a means, 
illustrated below, is provided for listing in digital storage the 
links between text strings and location .addresses. This, the shot 
list, is composed of three parallel lists whioh correlate a shot or 
portion of film, as denoted in the shot-endpoint columns, with a 
string of text In the text-string list, numbers in the shot end¬ 
point lists denote points In the videotape point location list 
representing respectively start and finish of a shot or significant 
portion of film, and the number in the corresponding position of 
the text description list represents the textual string (as enterec 
in the text-string list) with which the shot is correlated. 

SHOT LIST 

SHOT ENDPOINTS TEXT STRING DESCRIPTIVE 

start of end of OP SHOT 

shot on shot on 

tape tape 
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To provide a means of selecting shots, a viewing means 
Incorporating also a selection facility is included in the system, 
as illustrated in PIG. 4* In this description list, the oomputer 
displays on the television screen 68 an array of textual descrip¬ 
tions 69, 70 , 71 whloh have been indexed through the shot-list 
and text-string list from their locations on disk 21. 

To provide a means whereby shots ohosen for assembly into 
sequence may be kept in mind and rearranged, a viewing means for 
identifying shots already chosen is provided by this system, as 
seen In FIG. 5« The screen 72 functions as the rearrangement 
buffering device for temporarily holding tentative assembly lists. 
The description list 73 oontalns textual descriptions 74, 75, 76 
whioh are if selected transferred to assembly list 77 as seen in . 
example 78. 

To provide a means for internal accommodation within the 
digital oomputer system of the assembly lists thus created, a form 
of listing these materials is needed. Such a form is given below, 
in the assembly list storage list. 


ASSEMBLY LIST STORAGE LIST 


shot no. 

text 

point no. of 


string 

start trim point 


point no. of 
end trim polibt 


This list provides correlated positions of storage wherein are 
contained indexes of the starts and endpoints of the different 
shots. 

To provide a means of surveying the available shots 
matching a portion of the script, a sorlpt map facility Is providec 
as may best be seen in PIG. 6. On the screen 79 appear lines and 

_ to, tl, St, 33 

directions in the sorlpt^visually correlated with shot descrlptlonm 
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84, 85, 86 , 87 » 88, 89 and lines of variable length representing 
those shots 9P, 91, 92, 93, 94. 

To provide a means for the storage of information 
required for the display of the script map as in FIG. 6, a storage 
means is required as shown in the format below. This is a sohem¬ 
atic drawing of the script lls^j, ^showing the indirect method of 
listing in digital storage the location addresses of sorlpt lines 
and descriptions and their correlated shots and videotape points. 

SGRIFT LIST STORAGE- 

sorlpt film film item 

string information identity 

code 

« • • 

• • • 

• • • 

string no. n point or 1 

shot no. n 

string no. n+1 point or i 

shot no. n+1 

• e • 

• e • 

• • • 

Lines and directions in the sorlpt are stored as Indexed strings 

referred to by reference to their positions on the text-string 

list, as are shot description strings. The film item identity 

code is as follows: 1 = shot start, 2= shot end, 3= single point. 

Script strings are stored in the script string list, corresponding 

film information units are stored on the film information list cor 

related with the script string list, Identity codes are stored on 
ifiWh 

the fllm A ldentlty code list correlated with the film Information 
list, proportion numbers are stored on percentage-of-script-strlng 
list correlated with the film information identity code list. The 
nth position in eaoh list is oooupled by the item known to the 
oomputer as the nth item on that list, the ( n+-l )th position on 
eaoh list is occupied by the item known to the oomputer as the 
( n-vl )th item on that list, etc. The film information in the film 


percentage o; 
script 
string 

xx 

XX 
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information Hat is a point from the point list, correspondingly 
enooded in the film item identity code list with a 1 if it is a 
shot beginning, 2 if it is a shot end, and 3 if it is neither of 
those. The number in the peroentage-of-sorlpt-string list repre¬ 
sents the user's assignment to the Indicated point of approximately 
where in the script string the point falls, a parameter required 
for the relative positioning and lengths of the shot-lines in the 
script map (PIG. 5). 

To provide to the editor a graphical indication of trim¬ 
ming alternatives, the system inoludes a facility for displaying 
a “trim map," as seen in PIG. 7, which likewise uses screen 95 as 
a control meohanlsm. Two shots are schematically Illustrated 96, 
97* the leading shot 96 being distinguished by an arrow tail and 
the trailing shot 97 having an arrowhead. The two shots are shown 
annotated with the user descriptions of those two shots 98, 99. 
Points in the shots are shown as having been marked by the user 
100-111. Annotated shot portions 105-106, IO 7 - 1 O 8 , 110 -m are 
displayed with their annotations 112, 113, 114. Another shot-point 
109 is annotated 115. Other non-annotated shot-points 100, 101, 
102, 103 , 104 are shown with contemplated trim-lines 105 , 106, 107 , 
108. 

To provide a means whereby prospective trim points may 
be indexed for reference and for modification, and conveniently 
displayed ln*the trim map of FIG. 7* a trimming list is provided 
on the disk file, as may best be seen below. What follows schem¬ 
atically representBthe trim list, showing the method of listing in 
digital storage the trim addresses associated with a series of shot 
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TRIM LIST 


*■# 


10 


15 


20 


25 


30 


35 


location In list 


(o+l) 


trim address no. 


ADDRESS n 
ADDRESS n+1 


This list gathers together videotape points as -listed In 
the videotape point list In the sequence required for display of 
trim alternatives. The nth address Is occupied hy the looatlon on 
the videotape point list of what the computer knows as trim addres^ 
n, the (n-H )th address Is oocupied "by the location on the videotape 
point list of what the computer knows as trim address m-1 . eto. 

To provide a sequential record of the sonic constituents 
of a portion of the sound required In digital storage, a sonio con*f 
stltuent list Is provided, as will he seen below. 

SONIC CONSTITUENT LIST 


time division 
(mllllseoonds) 


t 

t<-l 


amplitude of frequency components 
(128 frequency divisions) 


(128 numbers 
(128 numbers 


This table Is a schematic drawing of the sonic constituent list, 
showing the method of listing sonic constituents In digital compute 
storage. Times t are recorded In the time-division list, and the 
values of 128 frequency components are recorded in correlated posl-j- 
tions of the amplitude list. 

Furnishing the editor with means for altering frequency 
constituents of a recorded sound, the system has a sonic constituent 
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screen display provided by the computer, best seen In PIG. 8. An 
array of 64 x 64 dots Is displayed on the screen 142, eaoh dot 
having eight degrees of brightness (Including extinction). The 
brightness of a point represents the comparative amplitude of a 
particular frequency (vertloal dimension) at a particular time 
(horizontal dimension). The sound maybe reconstituted through 
sonic restructuring unit 65 and played out through user loudspeaker 
66, at which time'the Image of frequency-points displayed 110 
moves past the now-llne 111. The sonic output from the sonlo re- 
oonstitutlon unit 65 at a given Instant exactly reproduces the 
frequenoy-volume oharaoterlstics specified by the oolumn of dots 
nearest the now-llne at that Instant. 

To furnish the editor with similar but simplified means 
for synchronizing analyzed sound with existing film Images, a 
sync map Is provided as may be seen in PIG. 9. Like the sonic 
constituent screen display In PIG. 8, the sync map uses the screen 
112 to display a visual presentation of sound 113 which the user 
may move in relation to a now-llne 114 while hearing the output 
through sonic reconstitution unit 65 and user loudspeaker 66. 

In this case only some measure of the overall volume is Indicated 
In the visual presentation of sound 113, but the user may annotate 
it with a text string 115 representing the dialog to be heard In 
the sound. Markings 116, 117, 118, 119, 120, 121, 122, 123 are 
also added by the user to indicate points of stress or mouth closure 
In speech, or other events In the audio. 

To provide a sequential plan for computer reference in 
adjusting audio levels, an audio cue list Is provided, which may 
be seen below. This shows the method of listing the audio oues In 
digital computer storage. 


- 16 - 


AUDIO CUE LIST 

timing action 

(milliseconds) 

change audio setting of 
tape #_ to _. 

change audio setting of 
tape #_ to _ 

• 

The adjustments to he made In audl^o playback from different units 
are listed In the action list while the instants at which the ad¬ 
justment is to he made are listed In the timing list. 

Furnishing an overall view of the separate sources of 
sound for a final audio mix, the invention provides an audio mix 
map, as will he seen In FIG. 10. This overview oomblnes on the 
screen 124 views of different sound-sources 125 , 126, 127 , as 
previously seen, moving In the present Instance together past 
a now-llne 128 in conjunction with the mixture of these sound 
sources In playback or recording. The user may similarly control 
their movement past the now-line In order to make adjustments. 

To provide a means for splitting one audio channel into 
two or more for stereophonic effect, or for assembling two or more 
audio channels from portions of others, the system provides 
separate-channel osonlo constituent lists, as above, for each of 
the channels to be thus assembled. 


x:xx:xx 

y : yy : yy 
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15 


20 


25 




Operation 

In use, the foregoing described apparatus may be employed} 
In practicing the method of this Invention to edit videotapes and 
|films and mix and match their soundtracks* 

To operate the system, videotape footage* original or 
copied from film. Is placed on VTR 22 and the controlling computer 
program is initiated* The user directs the presentation of the 
videotape on screen 1, and Its motion forward or backward, by meant 
of oontrol wheel 4. The motion of this oontrol wheel reaches the 
computer through a shaft encoder and 1 b analyzed by the computer 
program. Depending on the wheel's movement, the computer directs 

or- kaekward 

the forward A motion of source VTR 22 at full speed, reduced speed, 
or halted viewing. 

If tie viewer wishes to change scenes , he presses SELECT 
button 15, and receives a sequence of presentations in the current 
mode. Upon reaching the desired part of the production he presses 

(the MARK A HD ANNOTATE:button 7, thereby causing the select funotioi 
to end. 

The leftmost function buttons MARK & AHNOTATE 7, SCRIPT 
MAP 8, ASSEMBLE 9, TRIM 10 and AUDIO 11, are the principal defines 
of the activity state of the system each Initiating a state des¬ 
cribed by Its name. The five functions MARK & ANNOTATE, SCRIPT MAl|, 
ASSEMBLE, TRIM and AUDIO are Interlocked so that the user using om 
of the functions In relation to a specific scene may at any time 
switch to another of the functions, which will then relate to the 
same scene. Slnoe materials stored In the system tend always to 
have a sequence, It Is permissible for the user to enter any of 
these states from another and oontlnue to work with approximately 
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the same material in approximately the same part of the corpus of 
his videotapes. 

The MARK & ANNOTATE function permits points and portions 
of footage to he marked and annotated. When the user reaches a 
point on the footage which Is of interest to him, he halts the 
videotape and designates the point of interest by cuing pushbutton 3 

The oomputer program’s response to the pushbutton signal 
Is to examine the current address of the operating VTR from the 
current address line 45-49, and store that address In a table in 
the videotape point address list. As that specific frame on the 
videotape Is thenceforth identified to the computer as a marked 
point, whenever the same footage Is shown the computer flashes at 
that frame a cuing mark on computer-to-video display 64 whloh is 
then seen on the screen 44. 

The user may create an annotation to such a videotape 
address point, or create an annotation between two videotape address 
points, thus defining a shot or portion of film. To annotate a 
point in the videotape he presses POINT 12, after stopping the 
videotape being viewed at the desired point; to annotate a shot 
he presses SHOT after stopping the videotape being viewed at one of 
two desired endpoints, then moves that videotape to the other end¬ 
point, which must have been previously defined by means of MARK & 
ANNOTATE/POINT. To annotate a sequence, the user presses MARK & 
ANNOTATE, followed by SEQUENCE 14. The sequences associated with 
the region wherein he is working will then be available, first as 
assembly lists (to be discussed below) which he may name or, if he 
turns control wheel 4, as sets of shots which he .may View, in which 
case the sequence will be defined as the next two locations at 
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locations at which he presses button POINT 12. 

To create an annotation, the user types a textual des¬ 
cription Into the typewriter keyboard 5* The characters are trans¬ 
mitted as parallel digital signals in the standard manner to digits 1 
computer 20, which stores them on disk file 21 and stores their 
string address on the text-string list. A text program then dis¬ 
plays the text by means of computer-to-video display 64 on tele¬ 
vision screen 44 by methods well known to praotitioners of the art. 
The user watches the television soreen 44 for confirmation of his 
typing as it takes place. If he desires to make any changes in the 
text he has typed, to correct errors or change phrasing, he presses 
editing buttons 12A, 13A, 14 a In order to perform operations of 
Insertion, deletion or rearrangement of characters or words upon 
the screen. He then effeots the desired editorial changes through 
the use of light pen and typewriter keyboard upon the television 
screen, as will be understood by practitioners of the art. 

When an annotation has been completed, the user so in¬ 
forms the computer by re-aotivltating oontrol wheel 4 or one of the 
function buttons 7-15- 

Lookup of materials in the system takes place through 
the SELECT funotion, as controlled by SELECT button 15. The 
SELECT function permits the user to obtain what has previously been 
labelled by means of MARK & ANNOTATE. By pressing SELECT 15 
followed by POINT 12, the user sees a list of the previously- 
annotated points. By pressing SELECT 15 followed by SHOT 13, 


the user sees a list of the 


SELECT? 15 
previousl; 


followec 


f the previously 
SEQUENCE 14, the 


-annotated shots. By pressln 
user sees a list of the 


asn^fMSth ught -pen 6 at a- specific 


element on the list thus brought up, and then specify a function 
from among function buttons MARK & ANNOTATE 7, SCRIPT MAP 8, 
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ASSEMBLE 9, TRIM 10, AUDIO 11. Work under the new function will 
commence at the portion of the corpus that has-been defined by the 
light-pen selection. 

Through the use of the SCRIPT MAP function, the user 
creates a visible correlation between the footage and a photoplay 
script. When the user has seen a shot he wishes to map to the 
script, he presses MARK & ANNOTATE button 7 followed by SCRIPT MAP 
button 8, causing the system to enter SCRIPT MAP state. A display 
of the script, which has already been entered in the computer and 
is stored on the computer disk file, then occurs. The control 
wheel 4 now controls the movement of the script up and down the 
screen. When the appropriate portion of the script is reached, the| 
user points with the light pen to the approximate point in the 
sorlpt to which the shot address most recently viewed corresponds. 
The computer program then prepares entries for the script list, 
causing the point address no. to be stored in the film information 
list, its item code to be stored in the film item identity code, 
and its percentage of script string to be estimated by the program 
on the basis of the point selected with the light pen and entered 
in that column. The computer then, through a computer program, 
emplaces a vertical line on the right-hand side of the screen to 
represent the shot, and shows as well the shot description, as 
illustrated in examples PIG. 6. 

Resequencing of shots in this system takes place through 
the use of the ASSEMBLE function, effected by the ASSEMBLE button 
Pressing the ASSEMBLE button 9 causes the system to enter 
ASSEMBLE state. This causes an assembly list to-appear on the 
screeji. Depending on whether the previous function was SCRIPT MAP 
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or SELECT, a visible part of the sorlpt map or the shot directory 
'y will remain on the screen# In PIG. 5 the assembly list is shown 
in conjunction with part of the shot directory. 

The editor may dictate the rearrangement of shots by the 
5 following method: pointing at descriptions of the desired shots in 

the lefthand description list 73 causes these descriptions to move 
to the righthand assembly list 77 through the Intercession of the 
ASSEMBLE oomputer program. To the assembly list as viewed on the 
screen corresponds an assembly list on the oomputer disk file 
1° (not shown), holding numbers designating entries on the shot list 

in the format shown on p. 12. 

To view the resequenced materials, the user presses the 
SELECT button while in ASSEMBLE state and is shown a succession of 
assembly lists on the screen. He moves through these using control 
15 wheel 4 until he finds the desired shot list. He then touches the 

lightpen to the description of the shot he wishes to view at the 
beginning. 

When a shot description is copied to the assembly list, 
it may at any time be repositioned in the righthand list by 
20 pressing MOVE 14 a and pointing to a position between descriptions 

already on that list. It will then be transferred in the display 
to the point designated, and the data entries on the corresponding 
oomputer list correspondingly rearranged. 

’ The user may perform the MOVE editing operation on the 
25 assembly list at any time, to change the order of the shot descrip¬ 

tions, the corresponding lists, and any resulting presentation. 

To carry out the rearrangement thus directed, the computer 
oonsults the location address list, described earlier, and calculates 
the lengths and positions of the shots on the assembly list. 
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By means of a sorting program, whose characteristics will 
be well known to practitioners of the art, the computer pre-plans 
the succession of videotape movements required to oopy the desired 
shots on videotape recorders 22-26 in the most efficient manner, 
in such a way that two consecutive shots are not on the same 
recorder. It then effects the required copying from VTR 22, by 
transferring the contents of the specified videotape recordings 
to the succession of recorders so specified in the correct sequence 
and position. 

When the transfers necessary for showing the sequence 
have been completed, the computer shows the sequence of shots re¬ 
quested. To make changes in the sequence of the shots thus viewed, 
the user calls the assembly list for the sequence and makes the 
desired changes by rearranging the shot descriptions on the screen 
with the MOVE button 14 a and the light pen. The computer then 
directs the copying of the specified shots in some sequence which 
permits their showing in the newly specified sequence. 

To make trimming adjustments at the beginnings and ends 
of shots, the system provides a function called TRIM MAP, invoked 
by TRIM button lCl The editor calls a trimming map by halting the 
videotape near the end or beginning of a shot and pressing the 
button TRIM 10. The button TRIM 10 communicates with a computer 
program whloh displays on the television screen 1 a trimming map, 
qB may be seen in FIG. 7. 

The trimming map, generated by a computer program, is a 
diagram representing two shots to be connected, and one or more 
possible interconnections of interest to the editor. The leading 
shot is represented pictorlally by the tall of an arrow 96 at the 
top of the screen, and the trailing shot is represented pictorlally 
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■by the head of an arrow 97 at the bottom of the screen. The shot 
descriptions 98, 99 are displayed on the screen with their respec¬ 
tive arrows 96, 97• Any other annotated points 100-111 are also 
dlagrammatlcally indicated and annotated 112, 113, 114. 

Any annotations of points within the shot are likewise 
displayed as in example 115. 

When the trimming map has been called by pressing TRIM 
button 10, the user may make fine trimming adjustments between 
shots. To indicate a cutting-point, or point at which to make a 
transition, the user presses the button INSERT 12A and points with 
light pen 6 at the representation of the arrow, at a point along 
the line proportionally representing approximately the place in 
the shot at which he would like to make the cut. A mark appears 
at that point on the line, as shown in FIG. 7 in examples 111-115, 
and the corresponding point is adjusted on the trim list. 

To indicate a cut he wishes to see, the editor then 
presses SEQUENCE 14. With the light pen 6 he points at one 
cutting-point on each arrow. Signals from the light pen go to the 
computer 20, which consults a program and then adds to the screen 
display a dotted line or transition-line 105 , 106, 107 , 108 betweer 
the marks representing the two cutting-points that have been thus 
specified together. 

Tb see a transition he has so specified, the user points 
with a light-pen at one of the transition-lines 105 , 106, 107 , 108 
and presses cue button 3. The computer consults the list of 
transitions, ascertains the cutting-points Involved, and displays 
the two consecutive shots from two VTRs, intercut in proportion to 
the marks on the arrows. 

The user may adjust a transition by pressing the MOVE 
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"button 14 a, pointing with the light pen 6 at the point In a shot 
which is to be moved, and moving the light pen to the new point 
on the shot-line. The computer program receives the signals from 
the buttons and light pen, correlates them with the computer 
program, adjusts the position of the cuttlng-polnt on the point 
list, and modifies the display to show the transition-line In Its 
new position. 

Fades and dissolves are specified in the trim map by 
marking a cut-line with the light-pen after pressing INSERT 12A, 
and typing on keyboard 5 the letter F (for fade) or D (for dissolve 
and a number preceded by or containing a decimal point, which spec¬ 
ifies In seconds the fraction of a second or longer period of time 
over which the fade or dissolve is to occur. 

If the Improvement of recording quality Is needed for 
speech or sound stored on audio recorder 52 or Instrumentation 
recorder 51, or sound tracks of videotape recorders 22 to 26, this 
audio output is played under oontrol of the computer 20 by control 
lines 53, 54, 27-31 through switch and attenuator matrix 32 to 
sonic analyzer 62, which performs an analog-to-dlgltal conversion 
and a Fast Fourier Transform to obtain the frequency and amplitude 
components of the sound, which pass In digital form through line 
63 to computer 20 which stores them on disk 21 In the sonic 
constituent list, shown earlier. 

The user Invokes this function by pressing AUDIO 11 
and modifies the sound by pressing the MODIFf SOUND button 14 b, 
whereupon a sonic display sueh as that in FIG. 8 appears on the 
soreen. Using control wheel 4 the user causes the sonic display 
to travel through the now-llne 143, and hears concurrently its 
regenerated form through sonic reconstruction^^ and speaker 66. 
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Using the light pen 6 he may modify the display, pressing the 
light pen 6 to the screen once to decrease the value of a point 
indicated and twice to increase it. By this method he may remove 
sounds (such as background hiss or an automobile horn) which are 
visible through such a display but not removable by automatic means 
or he may by trlal-and-error improve the recording, as for instance 
by adding "realism" to a poor voice recording by accentuating 
higher frequencies characteristic of a particular speaker. 

If lip-synchronization is desired between film and 
recorded sound on some recording unit 51, 52, 22-26, the user 
presses SYNC button 13B and the sound needful of synchronizing is 
played through sonic analyzer 62, which performs an analog-to- 
dlgita.1 conversion and a Past Fourier Transform to obtain the 
frequency and amplitude components of the sound, which pass in 
digital form through line 63 to computer 20, which stores them bn 
disk 21 in the sonic constituent list shown earlier. 

A sync map, as shown in PIG. 9, is then created by a 
computer program, showing the average volume contour 113 of the 
converted sound. The user may employ control wheel 4 to move this 
display under now-line 114 and concurrently hear it through sonic 
reconstruction unit 65 and speaker 66. The user may annotate it 
with typewriter keyboard 5, which causes each character typed to 
appear under the now-line as an annotation 115 and thenceforward 
to remain in the same relative position to volume contour 113. 

The user may also create sync points (116-123) indicating points 
of stress or other features in speech or sound. Upon pressing 
the space bar of keyboard 5, the user now obtains the visual 
presentation of the videotape scene which he last saw, which he 
may move through at varying speed using control wheel 4. At slg- 
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nlflcant synchronization points In the videotape, such as explosions, 
door slams, mouth closures, wldest-mouth positions, etc., the user 
presses oulng button 3. The computer lists these cue-points and 
correlates them with sync-points 116-123 established earlier. 

By pressing the space bar of keyboard 5, the user causes 
the system to replay the scene, reconstituting the sound through 
sonic reconstruction unit 65 but In a timing which hastens or 
retards the designated sync points of the sound In exact synchrony 
with the cue-points specified by the user in the footage, all with 
no modification of the reconstructed frequencies and with smooth 
lnterpolatlve stretching or compressing of the time periods so 
specified. The user may alter the sync-points and cue-points 
respectively with buttons INSERT 12A, DELETE 13 A and MOVE 14 a until 
satisfied with the resulting synchronization* 

If it is desired to synchronize one recorded sound or 
utterance with another recorded sound or utterance, the two having 
corresponding parts (as for example one being a poor-quality 
recording made In synchronization with a motion picture, the other 
being a high-quality wild recording from the same script) the user 
may call both to a sync map and mark their corresponding parts. 

The timing of either can then be applied to the re-constltuted 
timing of the other. 

If It Is desired to create more than one audio channel 
from the available recordings, the visual sound-control facilities 
are used In two ways. By the use of light pen 6 on sync maps 
such as that in FIG. 9> one for each channel of audio desired, he 
may create oulng points such as 116-123 for each channel, permit¬ 
ting the digitized sonic constituents to be reconstituted in the 
desired synchronization. Such oulng points may be annotated with 
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symbols such as musical notes, as well as the text mentioned 
earlier. By the use of light pen 6 on sonic constituent displays 
such as that In FIG. 5, he may Indicate certain areas of the sonic 
cloud, and direct the machine to copy or transfer to the respective 
sonic constituent lists for each channel, of the type shown on 
p« 15, the components of sound so indicated with the light pen. 

With the light pen he may also divide sonic constituents, by 
pressing the light pen to a dot once for each level of intensity 
he wants transferred to a corresponding position on a separate 
channel. This enables a retroactive division of the sound into 
separate channels. 

The user may obtain automatic adjustment of intensity 
and phase of the separate audio channels to be reconstituted by 
creating on the screen with light pen 6 a diagram of the virtual 
positions, in an imaginary physical arrangement, of the sound 
source assigned to a channel, and the virtual positions of the 
separate imaginary microphones. The system is programmed, employ¬ 
ing means well understood in the field of frequency analysis and 
synthesis, for the automatic re-channeling, with appropriate volume 
and phase adjustments, of the sound as it would have been recorded 
in the setup he has diagrammed. By Indicating in the diagram the 
virtual positions of imaginary walls, the user may cause the systen 
to add-to the reconstitution those echoes which the walls depicted 
would produce. 

When the desired show has been completed in its visual 
aspect, with respect to all the shots in their correct sequence and j 

cutting-points, the user may direct the mixing of the audio, in its [ 

respective volume, by tracing a diagram of the desired volume- 
settings of different sound components. By pressing the MIX button 





12B, he summons to the screen the audio mix map shown in PIG. 10, 
with horizontal lines representing the separate audio tracks which 
have been selected after pressing the SELECT button 15. Event- 
points and annotations associated with the shots are shown also* 

To establish mixing levels, the user presses the INSERT button 12A 
and draws with the light pen upon the screen, creating desired 
volume contours 125, 126, 127* These create corresponding audio 
cues in the audio cue list depicted earlier. The user may then use) 
oontrol wheel 4 to begin the audio output, and hear the mixed 
tracks together in the proportions specified by volume contours 
125, 126, 127* To modify the volume contours he may use the MOVE, 
button 14 a to empower the light-pen to raise or lower a portion of 
a volume contour. The complete mixed audio is played out or 
recorded under user command, and the component diagrams 125, 126, 
127 move concurrently beneath now-line 128 as this occurs. 

The final production is created as follows from the 
different lists within the computer 20 and disk 21: if the original) 
is a videotape, it is placed on one VTR and a second-generation 
copy is produced on another VTR. If the original is a film, 
computer 20 prints out a negative-cutting list on a peripheral 
printer (not shown) or creates an information file on a paper tape 
punch (not shown) or .other peripheral device which may be used in 
conjunction with an automated' optical printer to produce the film 
master automatically from the original without human intervention. 








